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ABSTRACT 


This thesis approaches the question of ‘How much testing is enough?” by 
formulating a model for the combat situation in which the weapon (e.g., missile) will be 
used. Methods of Bayesian statistics are employed to allow the decision maker to benefit 
from prior information gained in the testing of similar systems by forecasting the 
operational gain from acceptance. A Microsoft Excel V7.0 spreadsheet serves as the user 
interface, and Visual Basic for Applications, Excel’s built in macro-language, is the 
language used to produce the source code. The methodology accommodates two different 
tactical usages for the missile: a single shot, or a salvo of two shots. The missile might be 
acceptable if used in the two-shot salvo mode, but not in the single shot mode, and this 
would imply a greater cost per mission. In the end the missile might not be judged cost- 
effective as compared to a competitive system. If the model proposed is (or can become) 
adequate much can be calculated/estimated before any operational tests are made. This 


could assist in economizing on operational testing. 





THESIS DISCLAIMER 


The reader is cautioned that models and computer programs developed in this 
research may not have been exercised for all cases of interest. While every effort has been 
made, within the time available, to ensure that the models provide accurate results and the 
programs are free of computational and logic errors, they must be further validated and 
verified. The completion of these tasks is left for further research. Any application of 
these models and programs without additional validation and verification is at the risk of 


the user. 
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EXECUTIVE SUMMARY 


As a result of the end of the Cold War, the size and structure of the United States 
military continues to undergo serious scrutiny. Although this is a dynamic process, the 
exact outcome of which 1s unknown, it is clear that in the future there will be significant 
reductions and changes in force structure, and a subsequent decrease in requirements for 
the generation of large numbers of new weapon systems. 

This trend will impact Testing and Evaluation programs, in that fewer dollars will 
be allocated for the test and evaluation infrastructure and process. As with other aspects 
of the acquisition process, the amount of testing performed should be determined by a 
cost-effectiveness measure. This can be described as a balance between the expense of 
testing and the reduction in risk for fielding the system gained by testing. One of the more 
expensive components of weapon system testing is that which involves destructive testing 
such as firing missiles or detonating ordinance. Such actions are required to provide some 
verification of the systems’ operational effectiveness and reliability. The need for testing 
in an operational environment adds further to the expense and complexity of the system 
evaluation. Clearly, testing methods which balance the need for economical efficiency and 
statistical integrity must be explored in order for the war fighter to continue to receive 
systems that are both effective and suitable, with firm attention to cost and value added. 

This thesis presents a formal analytical process for arnving at a number, t*, of 
missiles (or other expendable items) to test out of a finite lot or ‘buy’ of m. It does so by 
estimating the operational combat utility of the missile, given acceptance. This means that 
testing focuses on adding operational value, rather than on simply reducing uncertainty, as 
simple hypothesis testing procedures tend to do. The methodology accommodates two 
different tactical usages for the missile: a single shot, or a salvo of two shots. The missile 
might be acceptable if used in the two-shot salvo mode, but not in the single shot mode, 
and this would imply a greater cost per mission. In the end the missile might not be 


judged cost-effective as compared to a competitive system. If the model proposed 1s (or 


AV 


can become) adequate much can be calculated/estimated before any operational tests are 
nade. This could assist in economizing on operational testing. 

The method proposed here is a suggestion and an approach; it is not a finished 
product. Various questions must be answered before the approach is practical. For 
example: how does one specify the prior (parameters) and the probability of successful 
opponent retaliation? Answer: from organizational experience with analogous systems, 
and from distilled expert judgment. Also, what to do with a system that is rejected (after 
testing an appropriate number of times and getting fewer than minimum number of 
successes)? The model does not attempt to address the choice of whether to end the 
program, or to look for particular faults that caused the deficiency and attempt to correct 
them. This choice is situation-specific, but if the system capability is needed and the faults 
are identifiable and rectifiable at reasonable cost then the latter course is attractive. 
Careful retrospective analysis of the test conditions is always important, whatever the 
outcome. 

In summary it is argued that some organized and defensible test planning and 
decision aiding process is needed by the Operational Testing community. The present 


approach 1s a proposed step on the path to filling that need. 


XV 


1. INTRODUCTION 


A. BACKGROUND 


As a result of the end of the Cold War, the size and structure of the United States 
military continues to undergo serious scrutiny. Although this is a dynamic process, the 
exact outcome of which is unknown, it 1s clear that in the future there will be significant 
reductions and changes in force structure, and a subsequent decrease in requirements for 
the generation of large numbers of new weapon systems. 

This trend will impact Testing and Evaluation programs, in that fewer dollars will 
be allocated for the test and evaluation infrastructure and process. As with other aspects 
of the acquisition process, the amount of testing performed should be determined by a 
cost-effectiveness measure. This can be described as a balance between the expense of 
testing and the reduction in risk for fielding the system gained by testing. One of the more 
expensive components of weapon system testing 1s that which involves destructive testing 
such as firing missiles or detonating ordinance. Such actions are required to provide some 
verification of the systems’ operational effectiveness and reliability. The need for testing 
in an operational environment adds further to the expense and complexity of the system 
evaluation. Clearly, testing methods which balance the need for economical efficiency and 
statistical integrity must be explored in order for the war fighter to continue to receive 


systems that are both effective and suitable, with firm attention to cost and value added. 
B. CURRENT THINKING CONCERNING UNCERTAINTY 


When a test is designed it is set up to answer a decision maker’s question such as, 
‘How good is this?” or ‘Is it better than the one we have?” The challenge for the test 
designer is to translate these questions into something that is testable. Typically this 
translation results in statistical hypothesis testing. A key consideration in this type of 
testing is the fact that the statistician will never tell you that the hypothesis has been 
proven correct. He will only tell you whether or not it has been rejected. That 1s to say, 


statistical hypothesis testing can lead to only two outcomes: the hypothesis is rejected, or 


the hypothesis cannot be rejected. Statistical hypothesis testing never directly confirms the 
hypothesis being tested. It is evident that the formulation or wording of the hypothesis is 
critical to the information the testing produces. Furthermore, conventionally applied 
hypothesis testing does not take into account gains from making correct decisions, or 
losses from making incorrect ones. 

Statistical hypothesis testing has another unattractive quality in these days of 
shrinking budgets in that it does not allow the tester to take advantage of pmor 
knowledge. Only the data gathered in the context of the particular test formally qualifies 
for analysis. Information available from other resources such as developmental testing, 
testing of similar systems, field experience and the like are formally disregarded. The 
reason this could be unattractive in today’s fiscal environment is obvious. Information 
gained from another similar system’s testing process may remove the need for at least a 


portion of testing of the new system and thereby conserve the relevant funds. 
@ ALTERNATIVE THINKING 


Bayes’ theorem gives a probabilistically-based rule for updating the degree of 
belief in a hypothesis H (i.e., the probability of H) given additional evidence E and 
background information (context) I: 

P{H|I}P{E|H, 1 


P{H|E,I} = fem (1.1) 


Note that all the probabilities are conditional. They specify the degree of our belief in 
some proposition under the assumption that some other propositions are true. This would 
seem to imply that Bayesian methods require the addition of more assumptions to obtain 
results, which is true. The most important assumption made is that of the value of the 
‘prior,” the P{H|I} term in the equation above. Here lies the rub. Prior probabilities have 
been not only ignored but abhorred by some classical statisticians. This has been due 
chiefly to the frequently subjective nature of the prior probabilities. In practice there may 
well be sufficient domain knowledge to specify a prior. In general, prior probabilities can 


be assigned to any unknown parameters involved in a formulation. After additional data is 


obtained these priors can be updated to a posterior. Clearly, Bayesian methods can take 
advantage of knowledge from other sources that is relevant to the testing, and thereby 


potentially save cost and time during the test and evaluation phase of system acquisition. 
D. THESIS OBJECTIVE 


This thesis employs a Bayesian framework in order to address a simplified version 
of a decision problem in destructive testing. It approaches the question of “How much 
testing is enough?” by formulating a model for the combat situation in which the weapon 
(e.g., missile) will be used. It then employs sequential Bayesian thinking to infer the 
testing level needed to achieve optimal expected gain. Questions of risk are also 


addressed. 





i. PROBLEM FORMULATION AND SOLUTIONS 


A. INTRODUCTION 


A new system 1s proposed for acquisition, possibly to replace or upgrade an 
existing system. This new system will be accepted provided it meets minimum 
requirements for effectiveness and suitability which will be demonstrated through testing. 
This testing will be performed to replicate the mission and environment to which the 
system is expected to be exposed, and the results of a single test will be binary: success or 
failure. For the purposes of this discussion we assume the system is a missile and that the 
testing 1s destructive. That is to say, the test will destroy the missile but the record of its 
performance will be retained. 

We begin with m missiles which arrive as a lot and some number, t (O<t<m), of 
them which are to be tested to determine whether or not the lot will be accepted. An 
optimization problem presents itself. Ifa small number of missiles (2 for example) are 
tested, less information is gained but a larger number of missiles is available to the fleet for 
use, given that the lot is accepted. Conversely, if a large number of missiles (m-2 for 
example) is tested, much more information is gained but only a few of the weapons are 
available for the missions for which they were designed (in this case, only 2). The 
intention of this paper is to provide a formulation that links the information gained from 
testing to the effect that gathering that information has on the missile’s future 
performance. In addition, the possibility of informing the decision as to how the missile 


could be “best” employed, or the choice of a shooting policy, will be discussed. 
B. UNCERTAINTY AND OPERATIONAL GAIN 


Assume that each of the missiles has the same, constant, and unknown probability 
of success, p. The decision as to whether or not the missiles should be deployed to the 
fleet will be based on the value of p. An option is to take t of the m missiles and test them. 
The number of tests could range from 0 to m; the latter would of course result in perfect 


knowledge for the lot of missiles but would leave no weapons for the fleet to employ. 


Models can be built to represent the missile as it would be used operationally. For 
example: when the weapon 1s launched at an opponent he may, if he is missed, return fire 
with a weapon of his own. Let vy be the value of a win, meaning a weapon hit on the 
enemy and v, be the value of a loss, a weapon hit by the enemy; q is the probability that the 
enemy will have a successful counterfire given missile miss. For simplicity, assume the 


value of q is known. The resulting gain function is, after t tests, 


G,(p, & m) = (m -t)lvwp —v,(1- p)q] (Cal 
and represents the total expected gain from fielding a missile with success probability p 
determined from t tests which leaves (m-t) possible engagements. The subscript on G, 
indicates that one weapon at a time was fired at the enemy. A second model, that 


describes the policy of firing a 2-weapon salvo, has gain function 


G,(p,t;m) = mst lv (1-0- p) )-v,(1- p)q| (2.2) 





iat 2 2 
ae v.(1-G- p) )-v,(1- p) a]. (224) 
Where | x | denotes the largest integer less than or equal to x. We use the approximation 


in the thesis. Note that a shoot-look-shoot policy is not evaluated here. The latter, while 
possibly more economical of fielded missile inventory, may well increase the chance of 


successful counterfire by the opponent (a larger q value). 
C. THE DECISION RULE 


V4 Probability of Success Known 

If the decision maker knows the value of p he might then opt not to test, meaning 
that t=0. He would then evaluate Gi(p, 0; m): if the result is sufficiently positive, 
(exceeds a gain threshold), he accepts the system and fields it, while if the result 1s not 
sufficiently positive he rejects the system, achieving a gain of zero. The acceptance gain 
threshold may be defined by a predetermined minimum value of gain or by a minimum 
improvement in gain relative to a preceding system. This use of gain functions is of course 


quite simplified but serves the purpose of illustrating the method and provoking further 


thought. In any case the decision whether or not to accept a system reduces to whether or 


not the system’s probability of success, p, meets or exceeds some minimum threshold, p. 


For example, in the case of the gain function for G;, it can be seen that in order for the 


system to have positive gain, we must have a p-value that exceeds p = v,q/(v,, + vq). 


2M Uncertainty in p: Bayesian Approach to Acceptance, Given a Test 

The decision to test t leads to the acquisition of data which is assumed to be 
summarized entirely as s successes (s = 0,1,2,...,t). These data can now be used in the 
development of a likelihood for p by use of a binomial model, and, if p has a beta prior, a 


beta posterior for p: 


l(p; S, t) = B(a’, B')p* "(1 p) (2-3) 
where a’ =a+s and B’=8+t-—s. The values (a,8) characterize the prior density for 
p. This is the classical conjugate prior setup which is standard; see Appendix A for details 
of the derivation. If more investigation shows that an alternative prior is more appropriate 
it 1s straightforward to supply the necessary changes. Now a decision maker in possession 
of the posterior, equation (2.3), should use it to evaluate the expected gain. This is 
accomplished by computing the gain’s expected value with respect to the posterior 
probability distribution, equation (2.3). In the case of G;, the expectation is linear and of 


the form, 


EG, (p, ti m) 











S, t| = (m - )v.E[p S, t| - v,(1 - E|p S, t})q}, (2.4) 
representing the expected gain from a fielded system. An appropriate decision rule for this 


case 1s to field the system if E[G, (p, ts m) 





s,t] is positive; otherwise “reject” the system. 





This is mathematically equivalent to fielding the system if E[G,(p, t; m) S, t| 2 2, where 
g might represent the gain from utilizing an alternative system. Because of the form of Gi 
this is also equivalent to fielding the system if s(t)>s(t) where s(t) represents the 


minimum number of successes out of t tests which results in a positive value for G; (or a 


value of g>0). The derivation of s(t) for both the single-weapon and two-weapon salvo 


cases can be found in Appendix B. 


D. HOW MUCH TO TEST 


1. Single-Weapon Case 

The previous section indicates what decision to make given the test extent, t, and 
the number of resultant successes, s(t). Now take the position of the decision maker 
before any destructive testing is performed. He must consider testing to any level. That is 
to say, he must consider testing 1, 2, 3, up to m missiles and determining which of the 
values for t provides the best potential for gain from the remaining missiles. Of course the 
results of the testing must satisfy the decision rules previously stated for the system to be 
fielded. The prediction used depends on the binomial model and upon the prior, which is 


assumed to be the beta prior with parameters o and B: 


M(p) = B(a,B)p*"(1- p) (2.5) 


Conditionally upon p, 





P{ s(t) = § 


In order to predict s(t) simply remove the condition on p, using the prior; this encapsulates 


p, t} = a p (1 - p), ORS S51, (2.6) 


the decision maker’s uncertain knowledge at the time he must decide on t. Details are 


found in Appendix A. The resultant predictive distribution, b(s;t), 1s the beta-binomial 


t\( T(a+p) \( T(a+s)(P+t—s) 
v0 = Pfot)= 4) =() Reem) ree} = 


Now the expectation of gain, E[G,], depends on the value of s(t), which is unknown 
during the planning phase of the test but whose probability distribution 1s given by 
equation (2.7), a beta-binomial. Consequently, the value of future gain can be predicted 


by unconditioning on s: 


E{E|G,(p,t;m)}s, t]} =(m-t)|(v,, +v,q) >> xb(x;t)- vq > b(x;t) |. (2.8) 


x2s(t) x2s(t) 


This expected gain can be evaluated over the entire range of t in principle. 


2. Two-Weapon Salvo Case 

Maintaining the aforementioned position of the decision maker and the questions 
facing him, we address the case in which 2 weapons are fired sequentially with no delay 
between shots. The beta prior, equation (2.5) and predictive distribution for s(t), equation 
(2.7), remain unaltered from the single weapon case. Thus, the expectation of G> also 


depends on the value of s(t) and can be predicted by calculating 


- (B+t—x+1)(B+t—-x) 
—| M=t é a 
E{E[G,,(p.t:m)|sce),t} = I A |. Re, — (v., + wa) Zope b(x;t) 


(2.9) 
The derivation of s(t)can be found in Appendix B. This expected gain can also be 


evaluated over the entire range of t and consideration should be given to starting with 


small values of t and proceeding as in the single weapon case. 
Es RISK OF ACCEPTANCE 


The value of t, denoted by t*, which results in the maximum value of the predicted 
gain is determined by numerical search as described previously. Given that during the 


testing at least s(t) successes were observed and that the system was subsequently 


accepted, we are now interested in the risk associated with this acceptance. Two possible 
measures of this risk are (a) the predicted probability of kill per engagement (e.g., a 
successful engagement), and (b) the probability that the number of mission engagements 
result in fewer successes than some threshold, D. Such information supplements that 
given by the expected value of gain, which involves not just p but also the chosen costs or 


penalties vy , vj, and q. 


1. Single-Weapon Case 
The predicted probability of kill per engagement given acceptance of the system 
emerges from the now familiar beta-binomial distribution, conditional on acceptance of the 


system, s(t) > s(t) and with the beta based on the prior parameters, o and B: 


_ Lebo 
p(t) = == ——____ (2.10) 


>, b(s; t) 


s2s(t) 





Let M(t) be the number of successful engagements from utilizing the m-t missiles 
in combat. The probability that M(t) will fall short of some minimum standard, D, for kills 


can be found in terms of a binomial. First, 


P{ M(t) = k 





p,s} = Ba ‘Jer = al k=012....,1mst (2.11) 


Next, remove the condition on p using the beta posterior: 


m-t T(a+p+t) T(a+s+k\(B-stm—k) 
and the condition on s, given acceptance, yields after simplification, 
Z ( 
. rasp) 3 “Tas j+k\(B- j+m-k) 
j2s(t}\ J 
P{ M(t) = k|Accep tance} = SO  _ ene 
P(a+B+m) 53 | Ta j\'(B+t-3) 
j2s(t)\ )) 
Finally, 
D 
P{ M(t) < DjAccep tan ce} = > P{M(t) = k|Acceptan ce} (2.14) 
k=0 


Now the above, (2.14), can be evaluated at t = t* for various D-values. It may be judged 
that the probability of mission success is simply intolerably low despite the fact that 
expected gain is positive. In that case the acceptance criterion, s(t), may be adjusted, and 
attempts made to increase the intrinsic capability of the new missile system, which could 


encompass performance as well as tactical considerations. 
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2: Two-Weapon Salvo Case 

In this case the probability of kill per engagement is found by first deriving the 
expectation of the probability that both weapons fired in the salvo miss their intended 
target, (1-p)”. This expected probability of failure is found in a similar fashion to that of 


the expected probability of success using the beta posterior, described in the single- 


weapon case. The expected posterior probability of success is then FI - ¢ _ p) | Details 


are available in Appendix A. The predicted probability of kill per engagement, like the 


single-weapon case, is conditional on acceptance of the system, s(t) > s(t) and with the 


prior distribution with parameters, a and B: 
(B+t-s+1)(B+t-s) ) 
_ 2 oS OS) 


__ $2s(t) 
P(t) = > b(s:t) 


s2s(t) 


(2.15) 


Let M(t) be the number of successful engagements from utilizing the m-t missiles 
in combat. Since the missiles will be expended in pairs, let N(m,t) represent the maximum 


number of possible engagements with the missiles that remain after testing which is given 





: ae mak sod 
by the largest integer less than or equal to as denoted by a The probability that 


M(t) will fall short of some minimum standard, D, for kills can also be found in terms of a 
binomial: 


P{M(t) = klp,s(t)} = (Nim.t) (1-(1- p)*) (0 yr. k =0,1,2,...,N(m,t). (2.16) 


Removing the condition on p using the beta posterior (2.7) and the binomial theorem (See 


Appendix C), 


P{ M(t) = ks} = 





OjGR 


N(m,t))& a 2N(m,t)-2k+2n+P+t-s) 
ee 


ye a ) P(2N(m,t)-2k+2n+0+B+t) ; 


and the condition on s yields after simplification, 


P{M(t) = k|Acceptan ce} = 


kK (t\(N(m,t)\(k), ., P(a+j)P(2N(m,t)-2k+2n+P+t—j) 
> >" k (Sea [(2N(m,t)-2k+2n+a+P+t) 


j2s(t)n=0 
t) T(a+j\(B+t—j) 
xl) r(a+p+t) 


jes(t 


(2.18) 


, k = 0,1,2,...N(m,t). 


Finally, 
D 
P{M(t) < D|Accep tan ce} = » P{M(t) = k|Accep tan ce}. Cale 
k=0 


As with the single-weapon case, the above, (2.19), can be evaluated at t = t* for 
various D-values. It may be judged that the probability of mission success is simply 
unacceptably low despite the fact that expected gain is positive. In that case the 
acceptance criterion, s(t) may be adjusted, and attempts made to increase the intrinsic 
capability of the new missile system, which could encompass performance as well as 


tactical considerations. 


IW. NUMERICAL EXAMPLE 


A. INTRODUCTION 


Consider a lot of 30 missiles offered by a manufacturer who has performed 
developmental testing and is satisfied that the uncertainty in each missile’s kill probability 
per engagement can be described by a beta distribution with parameters « and B, equal to 
5 and | respectively. We assume (2.1) adequately models the system in its combat 
environment and that an expected gain of 0 or greater 1s sufficient for acceptance although 
a gain of 0 might indicate no improvement over a current system. We further assume the 
values for Vw, Vi, and q are known to be 1, 5, and 0.75. The 1 and 5 serve to indicate that 
the (negative) value of losing one of our aircraft is 5 times that of causing the destruction 
of one of the opponent’s aircraft. The 0.75 indicates that if the enemy has opportunity to 


counterfire he will with 0.75 probability succeed in shooting down our aircraft. 
B. THE ACCEPTANCE POLICY 
Accepting the system based upon a minimum expected gain is mathematically 
equivalent to accepting the system if s(t) > s(t). The value of s(t) is found by evaluating 
(B.6): 
s(t) = v\q(a +B +t) a 


Vy tVviq 
Po jeat) 
~-1+5(0.75) 
s(t)=33t-79 (3.1) 
Negative values of s(t)are set equal to 0 while all other values are rounded up to the 


nearest integer. This is consistent with both the idea that only non-negative numbers of 


successes can be observed and that s(t)is to provide a lower bound on acceptable gain 


which is itself restricted to values greater than or equal to 0. 


Next s(t) and (2.7) are used to evaluate the predicted expected gain given by (2.8) 
for each value of t (0<t<30). 


ELE|G, (p,t; m)|s(t), t|| Acceptan ce} =(m- HO +v,q) >> xb(x;t)- v,q p60) 


x2s(t) x2s(t) 


= (30- (679 S so] ee -3.75 > cw ed 


x2s(t) x x2s(t) x 
Relevant values of t, s(t), and predicted gain can be seen in Table 3.1. If the predicted 


gain were determined to be zero through the range of t, the system would be rejected 
without testing since there is at best no gain from the testing expenditure. Also, if the 


maximum expected gain is achieved with no testing, t = 0, then the evaluation of the risk 





Lag ae 7 oF 7.94 8.05 fb) (2 


Table 3.1. Example Results. 


associated with deployment of the system becomes more significant. Neither of these 
situations occur in this example. In this example maximum expected gain occurs when 2 
of the 30 missiles are tested and the remaining 28 missiles are deployed. The policy is to 
accept the missile if both of the tests are successful. It is noticeable that the expected gain 
has a somewhat flat or slowly-changing dependence on t, so a judgment that a few more 
tests are desirable, based on unmodeled considerations, does not incur a great change in 
expected gain. Extra tests might be conducted to reveal surprising system faults. 


Additional testing under different tactical conditions may very well be necessary. 
@: RISK ASSESSMENT 


We are now interested in the nsk associated with acceptance of the missile system 


subsequent to observing at least s(t) successes during testing. The first measure of risk 


considered 1s the predicted probability of kill per engagement given acceptance which can 
be obtained from (2.10) in concert with (2.7). The same numerical example is used for 


illustration. 








atpB+t g 


p(t) _ X2s(t) 


ey = re) -@) 
The second measure of risk considers the probability that the number of mission 


engagements result in fewer successes than some threshold, D given acceptance. This step 


1s accomplished by evaluating (2.13) 


m-t : | | 
| k [Tape a | Flesite)e (Biome 
P{M(t) a k|Acceptan ce} AO ae li 

[(a+B+m) =| f Patj\0 (B+t—j) 
j2 sit 


(3.4) 
8) r(8) | 7) r(s+x+k)F(31-x-k) 
(roxy 
(36) y “\r¢ +OG-x) 


then simply sum the results for the cumulative distnbution up to D. Table 3.2 contains an 





Table 3.2. Risk Summary. 


illustrative portion of the results for this example. In this example there is a 50% 
probability of 25 or fewer successes out of 28, or a 50% chance of 25 or more successes 


out of 28. Such information enhances the decision maker’s intuition. 


D: SUMMARY 


The example above demonstrates a formulation that links the information gained 
from testing to the effect that testing has on a particular measure of the performance of the 
system once fielded. Despite the fact that the example is simple, valuable information is 
gleaned from it. In this example some guidance is provided to an individual who might be 
deciding just how many of the 30 available missiles should be tested. While testing 2 of 
the 30 missiles may not be the answer, if one places some belief in the assumptions made 
at the beginning of the exercise it is apparent that the number to test is nearer 2 than a 
considerably larger number, such as 15. Furthermore, the expense has been minimal 
compared to that of expending weaponry under one set of tactical conditions. Additional 
missile testing under different conditions is needed to explore the range of possible combat 
environments, so it may be reassuring that a comparatively few tests are recommended for 


each of these explorations. 


IV. SOFTWARE IMPLEMENTATION 


A. INTRODUCTION 


The methods developed in previous chapters are, even for the simplest cases, 
computationally tedious. This invites one to use a computer not only for the convenience 
but also for the speed. Convenience is lost to some degree when one must learn a 
programming language in order execute the type of calculations involved in this 
formulation. On the other hand, more and more people are becoming familiar with 
spreadsheets and their use, especially in the analytical community. Increasingly, 
spreadsheets available commercially are suitable for situations that once required a custom 
program written in a traditional programming language. One such spreadsheet is 
Microsoft’s Excel V7.0. Excel and its accompanying macro language , Visual Basic for 
Applications, were used in the production of this project’s software. The code for this 
application is located in Appendix D. The ‘readme.txt’ help file and software installation 


directions are located in Appendix E. 
B. OVERVIEW 


1. Input 

The user is presented with a window which prompts him for the necessary 
information (See Figure 4.1). The inputs for both the single-weapon and two-weapon 
salvo cases are handled in an identical manner. Default values are indicated with the 
purpose of avoiding program crashes caused by blank input fields. Once the user is 
satisfied with his input selection, he merely clicks the “Begin” button or presses the return 


key on his keyboard to initiate the computations. 
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Figure 4.1. Input User Interface. 


2. Output 

After the user initiates the computations and the algonthm has executed, a 
spreadsheet page is produced. The worksheet contains a plot of the predicted gain values 
over the range of t (Figure 4.2) and several tables. The tables display information 
regarding the relationship between t, E[G;], and s(t); the optimal number of missiles to 


test, t*; and the values associated with the risk incurred if the system 1s fielded. 


Expected “Single-Shot Engagement’ Gain after Testing 

2.321429 
2.5 
2.444444 
2.272727 
2.045455 
1.958042 
1.794872 
1.582418 
1.339286 
1.078431 
0.808824 
0.553406 
0 281734 
0 





Expected Galr 





P{M=k} P{M <k} p bar(t) 

7.94E-06 7946-06 0.888689 
6 35E-05 7.14E-05 
0.000286 3.57E-04 
0.000953 1.31E-03 
000262 3.93E-03 
0.006287 1.02E-02 
0.013622 238E-02 
0.027245 5.11E-02 
0.051084  1.02E-01 
0.090815  1.93E-01 
0154386 3 47E-01 
0.252632  6.00E-01 
0.4  1.00E+00 


Max Gain - 
Optimal t: 
Minimum required successes 


kK 
0 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 


eo 
No 





Figure 4.2. Typical Output Worksheet. 
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Ge DEMONSTRATION 


The following serve to demonstrate the scope of this project’s software through 
various combinations of weapon cases and analysis outcomes. Although the combinations 
are not exhaustive, they illustrate sufficiently to express the breadth of outcome 
possibilities. 

1. Single-Weapon Case with Testing Recommended 

Consider a lot of 15 missiles whose prior probability of kill has a beta distribution 
with parameters a and B, equal to 5 and 1 respectively (Figure 4.2). The values of vy, v: 
and q are 1, 5, and 1 respectively. The values used for the other relevant parameters are 
displayed in the output worksheet for convenience. The algorithm finds that maximum 
gain is achieved when 3 missiles are tested and an acceptance policy of 3 successes from 3 
trials is employed. See Figure 4.2 for these results as well as the risk of acceptance 
predictions. 

Zz Two-Weapon Salvo Case with No Testing Recommended 

Consider a lot of 15 missiles whose prior probability of kill has a beta distmbution 
with parameters « and B, equal to 5 and 1 respectively (Figure 4.3). The values of vy, v; 


and q are 1, 5, and 1 respectively. The values used for the other relevant parameters are 


5 Expected "2 Weapon Salvo” Gain after Testing 
4.642857 


4321429 
3.97619 
3.625541 
32119t2 
2.914585 
2.56216 
2.199051 
1.836399 
1.472204 
1.1046 
0.738206 
0.369306 
0 


Expected Gain 
& w 
4 


OoOontmaarnnnaA WWHN =] O00 
i=) “> N w 


P{M=k} 
8.60E-05 


P{M < k} p bar(t) 
8.60E-05 0.952381 


Figure 4.3. 


Max Gain: 
Optimal t: 
Minimum required successes: 


NOOA WH = OC] *¥ 


0.000529 
0.002012 
0 006188 
0.017293 
0.047796 
0.147272 
0778823 


6. 15E-04 
2.63E-03 
8.82E-03 
2.61E-02 
7.39E-02 
2.21E-01 
1.00E+00 





Two-Weapon Salvo Case with No Testing Recommended. 
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displayed in the output worksheet for convenience. Note that the cost values are the same 
as those in Figure 4.2. However this case envolves consideration for the two-weapon 
salvo where the penalty for accepting a system with a low value of p is small. 
Subsequently, the algorithm finds that maximum gain is achieved when 0 missiles are 
tested and suggests system acceptance without testing (it is likely that this would be 
overridden in practice). In this case the nsk of acceptance analysis is particularly germane. 
See Figure 4.3 for these results. 

3: Single-Weapon Case with System Rejection 

Consider a lot of 15 missiles whose prior probability of kill has a beta distribution 
with parameters a and B, equal to 5 and 1 respectively (Figure 4.4). The values of vw, v; 
and q are 1, 25, and | respectively. The values used for the other relevant parameters are 
displayed in the output worksheet for convenience. The algorithm finds that maximum 
gain that could be achieved is no better than that gain which exists without the missile 
system being considered and suggests system rejection without testing. In this case the 


cost of loss is v; = 25 as compared to vy, = 5 in Figure 4.3. Acceptance of this system 


0 
Expected “Single-Shot Engagement” Gain after Testing 


ooo 0 00 0 0g 0g 0 0g 0 00 0 


t 
0 
1 
2 
3 
4 
5 
6 
7 
8 
) 
10 
14 
12 
13 
14 
15 


k P{M=k} P{M<k) p bar(t) 
REJECT SYSTEM 


Max Gain . 
Optimal t 
Minimum required successes: 





Figure 4.4. Single-Weapon Case with System Rejection. 


would result in gain which is at best zero and probably negative. In other words, 
acceptance of the system will place the decision maker in no better a position. See Figure 
4.4 for these results. 
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4. Two-Weapon Salvo Case with Testing Recommended 

Consider a lot of 15 missiles whose probability of kill has a beta distribution with 
parameters co and f, equal to 5 and 1 respectively (Figure 4.5). The values of vy, v; and q 
are 1, 35, and 1 respectively. The values used for the other relevant parameters are 
displayed in the output worksheet for convenience. Note that testing is recommended 
here (accepting the system if 5 successes are observed in 5 tests) while testing is not 


recommended Figure 4.3. In this case the cost of a loss here is v.= 35 as compared to v= 


Expected “2 Weapon Salvo” Gain after Testing 


1.055556 
1.136364 
1.101399 
1.007326 
0.884615 
0.824176 
0.735294 
0.612745 
0.470201 
0.316821 
0.15873 
0 


Expected Gain 


P{VM=k} P{M<k} p bar(t) 
5 416-06 5.41E-06 0.984848 
8.72E-05 9 .26E-05 


0.000846 9.38E-04 

Max Gain : 0.006875 7.81E-03 

Optimal t: 0.059094 6.69E-02 

Minimum required successes’ 0.933093 1.00E+00 





Figure 4.5. Two-Weapon Salvo Case with Testing Recommended. 


5 in Figure 4.3. It is reasonable that more care be taken in the present more costly case: 
the penalty for accepting a low value of p is much greater here than in the case of Figure 
4.3. The algorithm finds that maximum gain is achieved when 5 missiles are tested and an 
acceptance policy of 5 successes from 5 trials is employed. See Figure 4.5 for these 


results as well as the nsk of acceptance predictions. 


Pa 





V. SUMMARY DISCUSSION 


This thesis presents a formal analytical process for arriving at a number, t*, of 
missiles (or other expendable items) to test out of a finite lot or “buy” of m. It does so by 
estimating the operational combat utility, called gain, of the missile, given acceptance. 
This means that testing focuses on adding operational value, rather than on simply 
reducing uncertainty, as simple hypothesis testing procedures tend to do. The 
methodology accommodates two different tactical usages for the missile: a single shot, or 
a salvo of two shots. The missile might be acceptable if used in the two-shot salvo mode, 
but not in the single shot mode, and this would imply a greater cost per mission. In the 
end the missile might not be judged cost-effective as compared to a competitive system. If 
the model proposed is (or can become) adequate much can be calculated/estimated before 
any operational tests are made. This could assist in economizing on operational testing. 

The method proposed here is a suggestion and an approach; it is not a finished 
product. Various questions must be answered before the approach is practical. For 
example: how does one specify the prior (parameters) and the probability of successful 
opponent retaliation? Answer: from organizational experience with analogous systems, 
and from distilled expert judgment. Also, what to do with a system that is rejected (after 
testing t* times and getting fewer than s(t*) successes)? The model does not attempt to 
address the choice of whether to end the program, or to look for particular faults that 
caused the deficiency and attempt to correct them. This choice is situation-specific, but if 
the system capability is needed and the faults are identifiable and rectifiable at reasonable 
cost then the latter course is attractive. Careful retrospective analysis of the test 
conditions is always important, whatever the outcome. 

In summary it is argued that some organized and defensible test planning and 
decision aiding process is needed by the OT test community. The present approach is a 


proposed step on the path to filling that need. 
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APPENDIX A. BINOMIAL BAYES 


The probability of a number of successes given a fixed number of trials is 


calculated using the binomial distribution. The usual parameters are n, the number of 


trials; s, the number of successes in n trials; and p, the probability of success for any of the 


single trials. In the case of the beta-binomial distribution the parameter p is assumed to be 


random having a beta distnibution. After n trials have been executed determining s, the 


Bayes update of the beta prior for p 1s possible. 
A. THE BETA POSTERIOR 


Assuming a Beta prior for p, 
r(a+B) 


II,(p) - Torey? (lm p), <n = 120. a 0 <p: 


and binomial sampling (n trials, s successes), the likelihood function for p ts 


L(p;n,s) = [2 )p'( ~p) 
The Bayes posterior is found by multiplying the prior for p by the likelihood and 


normalizing 


a p(I-p)" “p*“(I-p) 


S 


II(p|n, s,.,B) = a 
J an ~p)**p*'(1-p)* dp 


0 


T(p n, $, OL, 8) =p (1 - p) 


Also, since the density must integrate to 1, 


B+n—s-l 





: T(a+B+n) 
T(a+s)F(B+n-s) 


Then, the beta posterior mean for p is 


E[p;a,|s,n] = 4 aay 


(A.1) 


(A.2) 


(A.3a) 


(A.4) 


(A.5) 


Later the expectation of (1-p)° will be needed. It can be derived in a fashion similar to 





that above 
E|(1- p)"|n,s] = B(a’,B’)((1- p) p*"(I-p)” dp (A.6a) 
a) Ta’ B’) oe 2 
E|(1 —p) In, s| = cone p (1 = p)” dp (A.6b) 


where a’ =a+s and B’=6+t-s. Finally, the mean beta posterior is 


(B+n—s+1\(P+n-s) 


(4+B+n+1)(a+B+n) 





(A.7) 


E|(1 —p) n,s| = 


B. THE PREDICTIVE DISTRIBUTION FOR THE NUMBER OF 
SUCCESSES 


This derivation takes advantage of the beta prior for p in the development of a 
predictive distribution for the number of successes, s, in t trials. A binomial model that 


utilizes the beta prior, (A.1), and that 1s conditional on p is formed 











Pfs(t)= ¢p.t} =(‘}p'(1- p) (A8) 
Removing the condition on p yields 
P{st)= 4 = [Jr o-n) Eo Pa (A9) 
Let w’ =a+sand B’ = 6+t-—s and combine terms to obtain 
P{s(t) = gt} = a “oR c p* "(1-p) dp (A.10) 
and simplify to get 
rlai= a} =[) TORO MC ES “— 
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APPENDIX B. DERIVATION OF MINIMUM SUCCESS THRESHOLD 


This appendix describes and then derives an expression for s(t), the minimum 


number of successes needed during testing in order to accept the system being tested. 
A. INTRODUCTION 


Consider a decision policy in which a system must provide a minimum level of gain 
from testing and subsequent fielding, given that the system is accepted. Let this minimum 


gain be g. Because of the form of the gain functions, (2.1) and (2.2), used in this paper 


acceptance of the system based on observing at least a minimum number of successes 
during testing, s(t),is mathematically equivalent to acceptance of the system based on 


receiving a minimum gain, g. 


B. DERIVATION OF s(t) 


As Single-Weapon Case 
For ease of discussion, a gain of 0 has been deemed the minimum value of gain 


which allows system acceptance in this paper, 
E|G,(p, t; m|s(t), t)| > 0. (B.1) 


From (2.4) it is evident that 
(m — |v. EfP|s) 1 ~ v.q( - E[p|s(t),1} >0 (B.2) 


implies that 


ies : 
Elpidt).t]2 oo 


Next substituting the beta posterior (A.5), 


oa + s(t) at Vig 


> (B.4) 
a+B+t vy,+viq 


and solving for s(t) gives 


Aa] 


rs . v.q(a +B +t) 
Vy yng 


Since s(t) > s(t) for non-negative gain it follows that 


=| Pd 


Von Vi 


where | z] is the smallest non-negative integer greater than or equal to z. 


a Two-Weapon Salvo Case 


(B.5) 


(B.6) 


Again for ease of discussion, a gain of 0 has been deemed to be in this paper, the 


minimum value of gain which allows system acceptance. 
E|G,(p, ts m(s(t), t)| ee) 


In a fashion similar to the single-weapon case, 
aot . - (v,, + vq) | (1 - p) |) = 0 


and substituting the beta posterior, (A.7), yields for the bracketed term 


(B+t-s+IMB+t—s) |. 
%, -(v,, rq) Pe 0. 





, see ee . ae 
(Note we are replacing E in (2.2) with (mt) r) 


2 
IEC 
ee v,(a+B+t+1\(a+B +t) 
v, +Vv\q 
and let 
r=(-s, 


Now rearrange (B.9) to obtain 


(B+r+i)(B+r)-A=0 
ho + (28 +1)r+(p? +B-A)=0. 
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(B.7) 


(B.8) 


(B.9) 


(B.10) 


(B.11) 


(B.12) 


The quadratic formula is now utilized to find the roots of (B.12). The root of 


interest, r*, will be the one that is real, positive and largest in magnitude. The roots can be 


found by computing 


— 2B -1+,/(28 +1) —4(p? +B - A) 


a (B.13) 
Finally, 
s(t) = max(t—| r° |,0) (B.14) 


where | x | is the largest integer less than or equal to x. 


Zo 





APPENDIX C. BINOMIAL THEOREM APPLICATION 


This appendix states the binomial theorem and discusses its application with regard 


to the derivation of a formulation for nsk assessment in the two-weapon salvo case. 


A. THE BINOMIAL THEOREM 
(x+y) = >o(B)xtye (C.1) 


Be APPLICATION OF BINOMIAL THEOREM 


Let M(t) be the number of successful engagements possible after t missiles have 


The 








been tested and let N(m,t) -|* |represen the largest integer less than —— 


probability of k successes out of N(m, t) engagements conditional on p, the probability of 
success for a single missile, and s, the number of successes during testing of t missiles 1s 


given by 





= kip, s} = ae ( —(1- p)’) (( — py (2) 





and reduces to 


P{ M(t) = k 





+e eape 1)" ( - ee a 


Removing the condition on p 


3] 


_ ees \S(ea(a : yy Sa eer aL py! C5) 


n=0 


ey r( 


a+s)I(B+t-s) 


T(a+B+t) V(a+s)l(2N(m,t)-2k+2n+B+t-s| 


at+sP(B+t-s) T(2N(m,t)-2k+2n+a+B+t-s) — 
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APPENDIX D. VISUAL BASIC FOR APPLICATIONS CODE 


This appendix contains the Visual Basic for Applications code used in the spreadsheet 


implementation software. 


A. G1.XLS, SINGLE WEAPON GAIN WORKBOOK 


" LT John R. Gorman 
' Spring 1997 
' Test Length Decision Analysis 


Sub get_inputQ 


‘This section gets input for vw, vl, gq, a, b, & m. The main subroutine 
‘is then called and executed. 


Dim DBoxBegin As Boolean 
DialogSheets("Input Box").EditBoxes("input_vw").Text = "1" 
DialogSheets("“Input Box").EditBoxes("input_vlI"). Text = "5" 
DialogSheets("Input Box").EditBoxes("input_q").Text = "1" 
DialogSheets("Input Box”").EditBoxes("input_a").Text = "5" 
DialogSheets("Input Box").EditBoxes(“input_b").Text = "1" 
DialogSheets("Input Box").EditBoxes("input_m").Text = "15" 
DBoxBegin = DialogSheets(“Input Box").Show 
lf Not DBoxBegin Then Exit Sub 
Sheets("Results").Range("vw").Value = 
DialogSheets(“Input Box"). EditBoxes("input_ vw"). T ext 
Sheets("Results").Range("vI").Value = _ 
DialogSheets("Input Box").EditBoxes("input_vl").T ext 
Sheets("Results").Range("q").Value = _ 
DialogSheets("Input Box").EditBoxes("input_q"). Text 
Sheets("Results").Range(“alpha").Value = _ 
DialogSheets("Input Box").EditBoxes("input_a"). Text 
Sheets("Results").Range(‘beta").Value = _ 
DialogSheets("Input Box").EditBoxes("input_b"). Text 
Sheets("Results").Range("m").Value = 
DialogSheets(“Input Box"). EditBoxes("input_ m").T ext 
Call plot_gain 
End Sub 


Sub plot_gain(Q 


‘Calculate expected mean gain for single weapon shot, display 
‘tables and plots. 


Application.ScreenUpdating = False 
Dim vw As Variant 

Dim vl As Variant 

Dim q As Variant 

Dim a As Variant 
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Dim b As Variant 
Dim t As Variant 
Dim s_bar As Variant 
vw = Range("vw").Value 
vl = Range(‘vi").Value 
a = Range(‘alpha").Value 
b = Range("beta").Value 
m = Range("m").Value 
q = Range("q").Value 
Sheets(‘Results").Select 
Range('t").Select 
Range(ActiveCell, ActiveCell.End(xiDown)).Select 
Selection.Clear 
Range('s_bar").Select 
Range(ActiveCell, ActiveCell.End(xlDown)).Select 
Selection.Clear 
Sheets("Results").Select 
Range("E_G1").Select 
Range(ActiveCell, ActiveCell.End(xIDown)).Select 
Selection.Clear 
Fort=OTom 
Range('t").Select 
ActiveCell.Offset(t, 0).Value = t 
S_bar = Application.RoundUp( _ 
Application.Max(0, (vl *q * (a+ b+ t) / (vw + vi *q)) - a)), _ 
0) 
EG1=0 
For j=s_bar Tot 
b_j1 = Exp(Application.GammaLn(a + b)) * Exp(Application.GammaLn(a + j)) * _ 
Exp(Application.GammaLn(b + t - J)) 
b_j2 = Exp(Application.GammaLn(a)) * Exp(Application.GammaLn(b)) * _ 
Exp(Application.GammaLn(a + b + t)) 
b_j3 = Application.Combin(tt, j) 
bj=b ji/b j2*b j3 
EG1_1= (we vi*q) * (at+j)/(a+b+ t)) - (VI * q) 
EG1 = EG1 + (EG1_1%* bj) 
Next j 
EG1 = Application.Max(EG1 * (m - t), 0) 
Range("E_G1").Select 
ActiveCell.Offset(t, 0). Value = EG1 
ActiveCell.Offset(t, 1).Value = s_bar 
Next t 
Dim check_range As Range 
Dim optimal_t As Integer 
Dim t_counter As Integer 
Dim optimal_gain As Variant 
Sheets("Results").Select 
Range("E_G1").Activate 
Range(ActiveCell, ActiveCell.End(xlIDown)).Select 
Set check_range = Selection 
Range("max_gain").Select 
ActiveCell. Value = Application.Max(check_range) 
optimal_gain = Application.Max(check_range) 
Range("E_G1").Activate 
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For t_counter=1Tom+1 
If ActiveCell. Value = optimal_gain Then 
optimal_t = ActiveCell.Offset(0, -1). Value 
GoTo loop_exit 
End lf 
ActiveCell.Offset(1, 0).Activate 
Next t_counter 
loop_ exit: 
Range("t_star"). Value = optimal_t 


‘This routine calculates the risks associated with accepting the 
‘system after testing. 


Dim k As Variant 

Dim PM1 As Variant 

Dim PM2 As Variant 

Dim PM3 As Variant 

Dim PM As Variant 

Dim t_star As Variant 

Dim m_t As Variant 

Dim i As Variant 

Dim p_bar As Variant 

Dim p_bar1 As Variant 

Dim p_bar2 As Variant 

t_star = Range("t_star’).Value 

m_t = m - t_star 

s_bar_star = Application.Max( _ 
(Application.RoundUp((vl * q * (a+ b+ t_star) _ 

/ (vw + vi * q)) - a, Q)), 0) 
Range(’s_bar_star').Value = s_bar_star 


p_bar=0 
p_bar1 =0 
p_bar2=0 


Sheets("Results").Select 
Range("PM”).Select 
Range(ActiveCell, ActiveCell.End(xlIDown)).Select 
selection.Clear 
Range(’k").Select 
Range(ActiveCell, ActiveCell.End(xlDown)).Select 
Selection.Clear 
Range("PMD").Select 
Range(ActiveCell, ActiveCell.End(xIDown)).Select 
selection.Clear 
Range(’p_bar”).Select 
Selection.Clear 
If ((t_star = 0) And Not (optimal_gain = 0)) Then 
GoTo NoTest 
Elself (optimal_gain = 0) Then 
GoTo Reject 


Else 

Fork=0 Tom_t 
PM2=0 

PM3 =0 


PM1 = (Application. Combin(m_t, k) *_ 


Exp(Application.GammaLn(a + b + t_star))) /_ 
Exp(Application.GammaLn(a + b + m)) 
For j = s_bar_star To t_star 
PM2 = PM2 + ((Application.Combin(t_star, j) * _ 
Exp(Application.GammatLn(a + j + k)) *_ 
Exp(Application.GammaLn(b - j + m - k)))) 
PM3 = PM3 + ((Application.Combin(t_star, j) *_ 
Exp(Application.GammaLn(a + j)) * _ 
Exp(Application.GammatLn(b + t_star - j)))) 
Next J 
PM = PM1 * (PM2 / PM3) 
Sheets("Results").Select 
Range("PM").Select 
ActiveCell.Offset(k, 0).Activate 
ActiveCell.Value = PM 
Next k 
For j =s_bar_star To t_star 
p_bar1 = p_bar1 + (a + J) * (Application. Combin(t_star, j) * _ 
Exp(Application.GammaLn(a + })) * _ 
Exp(Application.GammatLn(b + t_star - j)))) 
p_bar2 = p_bar2 + ((Application.Combin(t_star, j) * _ 
Exp(Application.GammaLn(a + j)) * _ 
Exp(Application.GammaLn(b + t_star - j)))) 
Next J 
p_bar = p_bar1 / ((a + b + t_star) * p_bar2) 
Range("p_bar").Select 
ActiveCell.Value = p_bar 
Range("p_bar_comment").Select 
ActiveCell.Clear 
Range("PMD").Select 
ActiveCell.FormulaR1C1 = "=RC[-1]" 
Fori=1Tom_t 
ActiveCell.Offset(1, 0).Activate 
ActiveCell.FormulaR1C1 = "=RC[-1]+R[-1]C" 
Next i 
Range('k").Select 
Fori=0O To mt 
ActiveCell. Value = | 
ActiveCell.Offset(1, 0).Activate 
Next | 
End lf 
GoTo Fini 
Reject: 
MsgBox "Reject System" 
Range("s_bar").Select 
Range(ActiveCell, ActiveCell.End(xlDown)).Select 
Selection.Clear 
Range("s_bar_star").Value = "NA" 
Sheets("Results").Select 
Range(‘p_bar").Select 
Selection.Clear 
Range("p_bar_comment").Select 
selection.Clear 
Range("PM").Select 


ActiveCell. Value = "REJECT" 
ActiveCell.Offset(0, 1).Activate 

ActiveCell.Value = "SYSTEM" 

GoTo Fini 

NoTest: 

MsgBox "Optimal Gain Achieved with No Testing" 
Sheets("Results").Select 

Range("p_bar").Select 

Selection.Clear 

ActiveCell. Value = a/ (a + b) 

ActiveCell.Offset(0, 1).Value = "<-- Prior probability used since 0 tested" 
Range("k").Select 

ActiveCell. Value = "NO" 

ActiveCell.Offset(0, 1).Activate 

ActiveCell. Value = "TESTING" 
ActiveCell.Offset(0, 1).Activate 

ActiveCell. Value = "DONE" 

Fini: 

Sheets("Results").Range("c2").Select 

End Sub 


‘This routine provides for the functions that occur on workbook opening. 


Sub Auto_openQ 

Call get_input 

ActiveWorkbook.Protect contents = True 
End Sub 


B. G2.XLS, TWO-WEAPON SALVO GAIN WORKBOOK 


' LT John R. Gorman 
' Spring 1997 
' Test Length Decision Analysis 


Sub get_inputQ 


‘This section gets input for vw, vl, g, a, b, & m. The main subroutine 
‘is then called and executed. 

Dim DBoxBegin As Boolean 

DialogSheets(‘Input Box").EditBoxes(“input_vw"). Text = "1" 
DialogSheets("Input Box").EditBoxes(“input_vl"). Text = "15" 
DialogSheets("Input Box").EditBoxes(“input_q").Text = "1" 
DialogSheets("Input Box").EditBoxes("input_a"). Text = "5" 
DialogSheets("Input Box").EditBoxes("input_b"). Text = "1" 
DialogSheets("“Input Box").EditBoxes("input_m"). Text = "15" 
DBoxBegin = DialogSheets("“Input Box").Show 

If Not DBoxBegin Then Exit Sub 
Sheets("Results”).Range(“vw"). Value = _ 

DialogSheets("Input Box").EditBoxes(‘input_vw"). Text 

Sheets("Results").Range(‘vl").Value = _ 
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DialogSheets("Input Box").EditBoxes(‘input_vl"). Text 
Sheets("Results").Range("q").Value = _ 
DialogSheets(“Input Box").EditBoxes("input_q"). Text 
Sheets("Results").Range(‘alpha").Value = _ 
DialogSheets(“Input Box").EditBoxes("“input_a"). Text 
Sheets("Results").Range("“beta").Value = _ 
DialogSheets("Input Box").EditBoxes("input_b"). Text 
Sheets("Results").Range("m").Value = _ 
DialogSheets(“Input Box").EditBoxes("input_m"). Text 
Call plot_gain 
End Sub 


Sub plot_gainQ 


' Calculate expected mean gain for single weapon shot, display 
‘tables and plots. 


Application.ScreenUpdating = False 

Dim r1 As Variant 

Dim r2 As Variant 

Dim r As Variant 

Dim vw As Variant 

Dim vl As Variant 

Dim q As Variant 

Dim a As Variant 

Dim b As Variant 

Dim t As Variant 

Dim s_bar As Variant 

vw = Range(‘vw"). Value 

vl = Range(‘vi"). Value 

a = Range(‘alpha").Value 

b = Range("beta").Value 

m = Range("m").Value 

q = Range('q").Value 

Sheets("Results").Select 

Range("s_bar").Select 

Range(ActiveCell, ActiveCell.End(xlIDown)).Select 

Selection.Clear 

Range("t").Select 

Range(ActiveCell, ActiveCell.End(xiDown)).Select 

Selection.Clear 

Range("E_G2").Select 

Range(ActiveCell, ActiveCell.End(xIDown)).Select 

Selection.Clear 

Fort=O0 Tom 
Range("t").Select 
ActiveCell.Offset(t, 0). Value = t 
rm =-b-05+05* (1+ ((4* vw* (atbtt+1)*(at+b+t)) /(vwt vi*q))) *0.5 
2=-b-0.5-0.5* (1+ ((4* vww* (a+ btt+1)*(atb+tt)) / (wt vi*q)))*0.5 
If (r1 <= 0) And (r2 <= 0)) Then GoTo Reject_t 
If (((r2 > 0) And (ri > 0)) And (r1 > r2)) Thenr=r1 
If (((r2 > 0) And (r1 > 0)) And (r2 > r1)) Thenr = r2 
If ((r2 > 0) And (r1 <= 0)) Thenr = r2 
If (r1 > 0) And (r2 <= 0)) Thenr =r 
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s_bar = Application.Max(0, (t - Applicatton.RoundDown(r, 0))) 
EG2=0 
Forj=s_barTot 
b_j1 = Exp(Application.GammaLn(a + b)) * Exp(Application.GammakLn(a + j)) *_ 
Exp(Application.GammaLn(b + t - j)) 
b_j2 = Exp(Application.GammaLn(a)) * Exp(Application.GammaLn(b)) * _ 
Exp(Application.GammaLn(a + b + t)) 
b_j3 = Application.Combin(t, J) 
bj=b_ji/b j2*b 43 
EG2_1=w- ((vwwt vi*q)*(b+t-j+1)*(b+t-j)_ 
tae tet ea + D'+:1))) 
EG2 = EG2 + (EG2_1 * b_}) 
Next | 
EG2 = Application.Max(EG2 * (m - t) / 2, 0) 
Range("E_G2").Select 
ActiveCell.Offset(t, 0).Value = EG2 
ActiveCell.Offset(t, 1). Value = s_bar 
GoTo next_t 
Reject_t: 
Range("E_G2").Select 
ActiveCell.Offset(t, 0).Value = 0 
ActiveCell.Offset(t, 1).Value = 0 
next_t: 
Next t 
Dim check_range As Range 
Dim optimal_t As Integer 
Dim t_counter As Integer 
Dim optimal_gain As Variant 
Sheets("Results").Select 
Range("B2").Activate 
Range(ActiveCell, ActiveCell.End(xlDown)).Select 
Set check_range = Selection 
Range("max_gain").Select 
ActiveCell.Value = Application.Max(check_range) 
optimal_gain = Application.Max(check_range) 
Range("b2").Activate 
For t_counter=1Tom+1 
lf ActiveCell. Value = optimal_gain Then 
optimal_t = ActiveCell.Offset(O, -1).Value 
GoTo loop_exit_4 
End lf 
ActiveCell.Offset(1, 0).Activate 
Next t_counter 
loop_exit_4: 
Range(‘t_star").Value = optimal_t 


‘This routine calculates the risks associated with accepting the 
‘system after testing. 


Dim t_star As Variant 

Dim s_bar_star As Variant 
Dim p_bar As Variant 

Dim p_bar1 As Variant 
Dim p_bar2 As Vartant 
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t_star = Range(‘t_star").Value 
m_star=-b-0.5+0.5%* (1 + (4 * w* (at+b+t_star+ 1) *_ 

(a + b + t_star)) / (vw+t vi *q))) * 0.5 
r2_star=-b-0.5-0.5* (1+ ((4* w*(atbt+t+1)*_ 

(a+b+t))/(vwet vi *q))) * 0.5 
If (((r2_star > 0) And (r1_star > 0)) And (r1_star > r2_star)) Then r_star = r1_star 
lf (((r2_star > 0) And (r1_star > 0)) And (r2_star > r1_star)) Then r_star = r2_star 
If ((r2_star > 0) And (r1_star <= 0)) Then r_star = r2_star 
If ((r1_star > 0) And (r2_star <= 0)) Then r_star = r1_star 
s_bar_star = Application.Max(0, (t_star - Application.RoundDown(r_star, 0))) 
Range('s_bar_star").Value = s_bar_star 
m_t=m-t_star 
N_mt = Application.RoundDown(m_t / 2, 0) 


p_bar=0 
p_bar1 =0 
p_bar2 = 0 


Sheets("Results").Select 
Range("PM").Select 
Range(ActiveCell, ActiveCell.End(xiDown)).Select 
Selection.Clear 
Range(‘k").Select 
Range(ActiveCell, ActiveCell.End(xIDown)).Select 
Sselection.Clear 
Range("PMD").Select 
Range(ActiveCell, ActiveCell.End(xIDown)).Select 
Selection.Clear 
Sheets("Results").Select 
Range('p_bar").Select 
Selection.Clear 
If ((t_star = 0) And Not (optimal_gain = 0)) Then 
GoTo NoTest 
Elself (optimal_gain = 0) Then 
GoTo Reject 


Else 
Fork =0 To N_mt 
PM1 =0 
PM2 =0 
For j = s_bar_star To t_star 
PM1ia=0 
Forn=0 Tok 
PM1a = PM1a + ((Application.Combin(t_star, j)) *_ 
(Application.Combin(N_mt, k)) *_ 
(Application.Combin(k, n)) * _ 
(een) 
Exp(Application.GammaLn(a + J)) * __ 
Exp(Application.GammaLn(2 *N_mt-2*k+2*n+b + t_star-j))/_ 
Exp(Application.GammaLn(2 * N_mt-2*k+2*n+a+b+t_star))) 
Next n 


PM1 =PM1+PMta 

PM2 = PM2 + ((Application.Combin(t_star, j)) *_ 
Exp(Application.GammaLn(b + t_star - J)) * _ 
Exp(Application.GammaLn(a + j)) / _ 
Exp(Application.GammaLn(a + b + t_star))) 

Next | 


40 


PM = PM1 / PM2 
Sheets("Results").Select 
Range("PM").Select 
ActiveCell.Offset(k, 0).Activate 
ActiveCell.Value = PM 
Next k 
Range("PMD").Select 
ActiveCell.FormulaR1Ci = "=RC[-1]" 
Fori=1ToN_mt 
ActiveCell.Offset(1, 0).Activate 
ActiveCell.FormulaR1Ci = "=RC[-1]+R[-71]C" 
Next t 
Range(‘k").Select 
Fori=O0ToN mt 
ActiveCell. Value = | 
ActiveCell.Offset(1, 0).Activate 
Next | 
For } = S_bar_star To t_star 
p_bar1 = p_bari + ((1 - ((b + t_star-j| +1) * (b+ t_star- j)/_ 
((a + b + t_star + 1) * (a+b + t_star)))) *_ 
Application.Combin(t_star, j) *_ 
Exp(Application.GammaLn(a + })) * _ 
Exp(Application.GammaLn(b + t_star - j))) 
p_bar2 = p_bar2 + ((Application.Combin(t_star, j) * _ 
Exp(Application.GammaLn(a + j)) * __ 
Exp(Application.GammaLn(b + t_star - j)))) 
Next | 
p_bar = (p_bar1 / p_bar2) 
Range('p_bar").Select 
ActiveCell.Value = p_bar 
Range(‘p_bar_comment'").Select 
Selection.Clear 
End lf 
GoTo Fini 
Reject: 
Range('t").Activate 
Fori=OTom 
ActiveCell.Offset(i, 0).Value = | 
ActiveCell.Offset(i, 1).Value = 0 
Next i 
Range("max_gain").Value = 0 
Range(‘t_star").Select 
Selection.Clear 
MsgBox "Reject System" 
Range("s_bar").Select 
Range(ActiveCell, ActiveCell.End(x!Down)).Select 
Selection.Clear 
Range("s_bar_star").Value = "NA" 
Sheets("Results").Select 
Range(‘k").Select 
Range(ActiveCell, ActiveCell.End(xIDown)).Select 
Selection.Clear 
Range(‘p_bar").Select 
Selection.Clear 
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Range("p_bar_comment").Select 
Selection.Clear 
Range("PMD").Select 
Range(ActiveCell, ActiveCell.End(xiIDown)).Select 
Selection.Clear 
Range("PM").Select 
Range(ActiveCell, ActiveCell.End(xlDown)).Select 
Selection.Clear 
ActiveCell.Value = "REJECT" 
ActiveCell.Offset(0, 1).Activate 
ActiveCell.Value = "SYSTEM" 
GoTo Fint 
NoTest: 
MsgBox "Optimal Gain Achieved with No Testing" 
Sheets("Results").Select 
Range('p_bar").Select 
Selection.Clear 
ActiveCell.Offset(O, 1).Value = "<-- Prior probability used since 0 tested" 
N_m = Application.RoundDown(m / 2, 0) 
Range("k").Select 
Fork=0ToN_m 
PM1=0 
PM2 = 0 
S_bar_star=0 
t_star=0 
For j = s_bar_star To t_star 
PMia=0 
Forn=0Tok 
PMia = PM1a + ((Application.Combin(t_star, })) *_ 
(Application.Combin(N_mt, k)) * _ 
(Application.Combin(k, n)) * _ 
CCU it) ae 
Exp(Application.GammaLn(a + j)) * _ 
Exp(Application.GammaLn(2 *N_mt-2*k+2*n+b+t_star-j))/_ 
Exp(Application.GammaLn(2 *N_mt-2*k+2*n+a+b+t_stan)) 
Next n 
PM1=PM1+ PMia 
PM2 = PM2 + ((Application.Combin(t_star, j)) *_ 
Exp(Application.GammaLn(b + t_star - j)) *_ 
Exp(Application.GammaLn(a + j)) / _ 
Exp(Application.GammaLn(a + b + t_star))) 
Next j 
PM = PM1 / PM2 
Sheets("Results").Select 
Range("PM").Select 
ActiveCell.Offset(k, 0).Activate 
ActiveCell. Value = PM 
Next k 
Range("PMD").Select 
ActiveCell.FormulaR1C1 = "=RC[-1]" 
Fori=1ToN mt 
ActiveCell.Offset(1, 0).Activate 
ActiveCell.FormulaR1iC1 = "=RC[-1]+R[-1]C" 
Next | 
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Range("k").Select 
Fori=OToN_m 
ActiveCell. Value = | 
ActiveCell.Offset(1, 0).Activate 
Next | 
For j = s_bar_star To t_star 
p_bar1 = p_bar1 + ((1 - ((b + t_star-j +1) * (b+ t_star-j)/_ 
((a + b + t_star + 1) * (a+b + t_star)))) *_ 
Application.Combin(t_star, j) * _ 
Exp(Application.GammaLn(a + })) *_ 
Exp(Application.GammaLn(b + t_star - }))) 
p_bar2 = p_bar2 + ((Application.Combin(t_star, j) * _ 
Exp(Application.GammaLn(a + j)) *_ 
Exp(Application.GammatLn(b + t_star - j)))) 
Next j 
p_bar = (p_bar1 / p_bar2) 
Range("p_bar").Select 
ActiveCell.Value = p_bar 
Fini: 
Range('G5").Activate 
End Sub 


‘This routine provides for the functions that occur on workbook opening. 


Sub Auto_open() 

Call get_input 

Active Workbook.Protect contents = True 
End Sub 
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APPENDIX E. INSTALLATION INSTRUCTIONS 


The installation diskette has the following three files: 

i G1.xls - the Excel workbook which contains the algorithm for analysis of 
the single weapon case. 

2, G2.xls - the Excel workbook which contains the algorithm for analysis of 
the two-weapon salvo case. 

3 Readme.txt - an ASCII text file which contains the text found in this 
appendix. 

To use the analysis files, simply open them with Excel V7.0 or later and follow the 
dialog box prompts. Input parameters can be changed and the algorithm repeated by 
clicking the button on the worksheet marked “Press to Change Parameters.” The 
workbooks open recommending a “read-only” format. This is done to protect the 
structure of the “Results” worksheet. The worksheets which contain the input dialog box 
and the source code are “hidden.” The source code is also “protected” and cannot be 
altered. If one wishes to alter the code to develop the ideas of the software further, cut 


and paste the appropriate code into a different workbook and proceed. 
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